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Introduction 



Over the past five to ten years, most states and many school districts have developed report 
cards or other types of documents to communicate information about student perfor- 
mance and program effectiveness to parents, policymakers, and other stakeholders. Much 
of this work has been driven by state accountability initiatives and recent reauthorizations of 
the Elementary and Secondary Education Act (ESEA), although some agencies have clearly 
made public reporting and accountability priorities of their own accord. With the 2001 
reauthorization of ESEA, known as the No ChildL^fiBehindA.ctof2001 (NCLB), federal 
accountability and reporting have been reinforced and expanded in several ways: 

• A number of specific indicators have been added to the sets that must be reported at 
the school, district, and state levels; 

• Disaggregated (subgroup) performance indicators must now be analyzed for 
accountability purposes; 

• Specific school-level accountability consequences are mandated for a school that fails to 
meet its improvement targets; 

• States education agencies (SEAs) and local education agencies (LEAs, which includes 
school districts, school systems, etc.) must produce their accountability reports for each 
school year by the beginning of the next subsequent school year; and 

• Reports must be both physically and linguistically accessible to a broad range of 
stakeholders. 

These expanded federal mandates, together with a variety of state and local accountability 
reporting requirements, are placing states and districts under escalating pressure to produce a 
greater number of increasingly complex reports at the state, district, and school levels — and 
to do so more quickly than ever before. They pose particular challenges for accountability 
reporting because these reports convey high stakes information for schools and districts and 
are expected to serve multiple purposes for multiple audiences. 

Generating such reports is not easy. In addition to substantial human and capital resources, 
report design, production, dissemination, and follow-up support require extensive expertise, 
the integration of multiple data management systems, and the coordination of often 
disparate poUcy perspectives. Further, state and local education agencies staff must simulta- 
neously juggle current practices, redevelopment activities, and transition strategies as they 
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comply with the new law. Thus, even though many agencies have already been publishing 
report cards or other types of accountability reports, most will face substantial challenges in 
meeting the new requirements — ^with few examples on which to draw. 

To assist state and local educators in these endeavors, the Council of Chief State School 
Officers (CCSSO) developed this monograph,^ Guide to Effective Accountability Reporting 
through the ASR SCASS (Accountability Systems and Reporting State Collaborative on 
Assessment and Student Standards). A Guide to Effective Accountability Reportingv& intended to 
serve as a resource for the staffs of state education agencies (SEAs) and local education 
agencies (LEAs) who are responsible for producing state, district, or school report cards of 
the type required under many state or district accountability systems as well as under NCLB. 
This guide is not intended to provide an academic discussion of the nature of indicators 
and indicator systems, nor is it meant to cover the broad territory of accountability issues. It 
is meant to provide a resource for agencies, and to spur the thought of practitioners, as 
accountability reporting systems are tooled to meet the requirements of NCLB. 

While each state or local agency will have different approaches and starting points for the 
development of accountability reports, and different end products, it is a good idea for all 
agencies to clearly document processes are used in report development. This wUl 
provide evidence that reports were developed in a sound manner, help to ensure the 
process is comprehensive in addressing all relevant reporting requirements, and facilitate 
subsequent redevelopment plans. 
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Organization of this Guide 
Chapter 1 — Identifying Reporting Needs 

This chapter is designed to help state and local education agency staff think through the 
goals and requirements of the NCLB required reports. It begins by delineating the reporting 
requirements under NCLB, then offers suggestions for how agencies can build or refine 
their reporting systems to meet these requirements effectively. 

Chapter 2 — Designing Reports for the No Chiid Left Behind Act 

NCLB prescribes specific content requirements for accountability reports. This chapter 
includes a list of NCLB reporting requirements, as well as examples of potential assessment 
and AYP reports that meet these requirements. 

Chapter 3 — The Design Process 

Because it is import not only to present the required accountability information, but to 
present it well, this final chapter offers an overview of the design process itself, including 
general suggestions related to text, graphics, and alignment of report cards with other 
documents in state and local accountability systems. 

Appendices 

• Appendix A provides details about additional non-public reports, NCLB definitions 
of highly qualified teachers and graduation rates, and program evaluation reports. 

• Appendix B provides a list of concepts designed to help guide the report design 
process. 

• Appendix C includes a variety of resource information on report design, examples, 
and other publications. 
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Chapter 1 : Identifying Reporting Needs 

Most states and many school districts already produce a number of reports meant to 
publicize information about student performance and program effectiveness to parents, 
policy makers, and other stakeholders. Whether the catalyst for developing these reports was 
a legislative mandate, a local commitment to transparency and accessibility, or grew out of a 
data-based approach to educational improvement, all are ultimately driven by a demand for 
more and better education information. Whatever the case, these reports reflect the 
system’s values to stakeholders: the indicators reported illustrate how well schools, districts, 
and the state are doing in several outcome and process domains and, implicitly, which 
outcomes are valued and which characteristics, inputs, and processes are believed to con- 
tribute to these outcomes. 

What to Report 

Recent expansions of state and federal accountability legislation, most notably the No Child 
LeftBehindActof2001 (NCLB), are now forcing state and local agencies to reevaluate their 
school and district reports as well as the accountability systems that include them. Though 
this will result in numerous challenges across the board, it also provides an oppormnity for 
agencies to simplify and enhance reporting systems. Specifically, rather than add new 
elements to existing disparate reports serving multiple purposes and audiences, 
NCLB requirements provide the opportunity and impetus for agencies to take 
stock of their current reporting systems, determine their goals and objectives for 
reporting systems, and then chart the path between these two points. ^4 Guide to 
Effective Accountability Reporting aims to assist state and local education agen- 
cies (SEAs and LEAs) meet these challenges. 

Accountability Report Cards as Part of an Agency's Larger 
Reporting Systenn 

A first step in designing a reporting system to meet NCLB requirements is to take stock of 
existing reports. NCLB has specific accountability reporting requirements for all states, 
much of it in the form of report cards. In many cases, SEAs and LEAs will find existing 
reports include some of the same indicators, cover the same domains, and will probably be 
produced by some of the same staff as those required by NCLB. (See CCSSO, 2002, for 
a list of all state reports and web links.) 
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However, accountability report cards may differ from other reports in several important 
ways: 



■ Unlike much agency reporting targeted to audiences of policymakers, experts, or 
other department staff, report cards target the needs of public stakeholders: 
parents, educators, community members, researchers, and others; 

■ Report cards convey a message with high stakes for individual schools or districts; 

■ Report cards make an agency’s values and expectations explicit; 

■ Report cards provide a descriptive and evaluative “snapshot” of individual schools 
and districts rather than a summary statement across schools or districts; 

■ Report cards are typically short and focused on student achievement, but they can 
encompass a broad range of indicators. They are more of an “executive sum- 
mary” rather than a traditional full report; and 

■ Even though they are summaries themselves, report cards should provide more 
detailed information about individual schools or districts than what can be included 
in a press release or reported by the media. These reports may also provide 
pointers to resources for additional information. 

While report cards and the accountability system that includes them are mandatory under 
NCLB, the formality and specificity of the NCLB accountability requirements suggest that 
SEAs and LEAs will need to reconsider any existing report cards and other accountability 
reports, both in order to reduce duplication of information and efforts and to create a 
cohesive system of accountability reporting. Simply producing a new set of report cards to. 
meet NCLB requirements while continuing existing reports is likely to be repetitive and 
confusing. 

NCLB Reporting Requirements 

NCLB reporting requirements encompass a wide range of reports. Figure 1 outlines three 
types of reports, and describes these requirements in relation to purpose, data included, 
agency responsible for producing them, and other reports that agencies are likely to be 
producing. Appendix A provides further details. 
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Figure 1 : Types of Reports Specified by NCLB 

Report Category General Description Typically Produced 



^y When 



Assessment Reports 


Summarize performance at the 
individual student, classroom, 
school, district, and state level 


A Contractor 


Before the 
beginning of 
the next school 
vear 


Accountability Reports 


Report cards or profiles at the 
school, district, and state levels 
that may be linked to state 
accountability systems; 

Each state must submit an 
annual performance report to 
the US Deoartment of Education 


State or district 
agency, with 
assistance 
from a 
contractor 


Before the 
beginning of 
the next school 
year 


Program evaluation 


Summarize activities and services 


in most cases. 


Annual or 


reports 


at the program level; also describe 
evaluation methods and criteria 
and include the results and 
consequences of the evaluation 


the state 
agency; 
districts are 
responsible for 
some reoorts 


Biennial 



Assessment Reports 

All states are required under NCLB to implement standards-based student assessment 
systems — most details of which are beyond the scope of this guide. Student assessment 
systems typically include several types of reports. 

In addition to these content mandates, NCLB directs states and districts to publish the 
results of the assessments before the beginning of the next school year. Accountability 
reports, which include a substantial amount of assessment information, must also be 
published prior to the beginning of the next school year. Currently, several states are 
changing to a fall testing window or moving testing to earlier in the spring to meet this new 
requirement (see CCSSO, SSAP, 2002). 
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Accountability Reports 



NCLB requires states to produce state report cards and to produce, or ensure that districts 
produce, report cards at the district and school levels. These report cards differ from the 
assessment reports in that they must include indicators of assessment performance as well as 
indicators of other, non-assessment features. They also differ from the program evaluation 
reports required by NCLB because the report cards are meant to convey information to 
stakeholders within the state. 

Three levels of report cards are mandated under NCLB. All three levels of report cards are 
required to be implemented beginning in the 2002-03 school year, reflecting 2001-2002 data, 
and to be produced and disseminated annually thereafter. Requirements for each level, basic 
information about the production and dissemination of these report cards, and the schedule 
of reporting, are listed in Figure 2. Further details on the NCLB reporting requirements 
can be found in Appendix A. 

Evaluation Reports 

NCLB requires state education agencies, and/or districts, to produce a number of evalua- 
tion reports for the programs the legislation encompasses. A full description of these 
evaluation reports is beyond the scope of this guide, though it is important to note where 
requirements for the accountability reports and requirements for the evaluation reports are 
related so that production of these reports can be coordinated and the information within 
them can be aligned. The report cards provide more detail at the state, district, and school 
level and the evaluation reports reflect more detail on the program level. For example, while 
the Annual Report to the Secretary requires information on the acquisition of English 
language Proficiency, results of the English Language Proficiency Assessments, the biannual 



Contractor versus Staff 

Hiring a contractor to produce reports does not mean that an agency has 
relinquished its responsibility for ensuring that the reports are accurate and of 
high quality. Regardless of how much money a contractor is paid, and how 
rigorous their quality assurance and review systems are, an agency should 
review and approve every report it creates or authorizes a contractor to create. 

or LEAS should require contractors to provide clients with raw data files that 
will enable them to build adequate time into their production schedules to 
allow clionts to conduct thoir own reviews. 
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Figure 2: NCLB Requirements for the Production and Dissemination of 

AccountabiliV Report Cards 

Issue Requirements for School, District, and State Report 



Who is reponsible for 
production and 
dissemination? 



How must the report 
card be disseminated? 



What other types of 
information must be 
made publicly 
available? 



School and District - The legislation specifies that the district is responsible for the 
school and district report cards, but the state may produce these on behalf of the district. 
If the district does produce these report cards, the state is still responsible for ensuring that 
they are indeed produced and disseminated and that they meet all of the stated 
requirements. 

State - The state education agency is responsible for producing and disseminating the 
state report cards. 

School and District - The district must disseminate school and district report cards to: 

■ all schools in the school district served by the local educational agency; 

■ all parents of students attending those schools; and 

■ the community, through public means, such as 

— posting on the Internet, 

— distribution to the media, and 

— distribution through public agencies 

State - No dissemination requirements are indicated either in the legislation or in the 
regulations published to date. However, the annual report that all states must provide to 
ED does summarize many of the elements of this report card. 

School - At the beginning of each school year, districts that receive Title I funds must 
alert parents that they may request, and the district must provide: 

■ the qualifications of their child's teacher, 

■ the level of achievement of the parent's child in each of the State academic assess 
ments. 

In addition, if the school has been identified for improvement, the LEA must provide an 
explaination of what the indentification means, and inform parents of the steps taken 
and services available to improve student achievement. 



*See Appendix A for more information and detaiis. 
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Title III report requires information how the assessments address the four cognitive de- 
mands of language. Appendix A provides an overview of the evaluation reporting 
requirements under NCLB . 

Howto Plan and Implement Reports 

Accountability reports provide the opportunity for SEAs and LEAs to articulate to a broad 
audience — ^locally and nationally — ^valued outcomes, and also the characteristics and prac- 
tices that are believed to contribute to them. This contrasts with the compliance— monitoring 
roles of SEAs and LEAs of days past, in which reporting was about schools and districts 
providing evidence that they were meeting process requirements, even if they weren’t 
particularly effective in improving student learning. Thus the new accountability reporting 
paradigm provides the opportunity to utilize accountability reports as a statement of an 
agency’s approach to educational improvement. They can highlight successes, illustrate 
results, and convey to stakeholders agencies’ plans to meet challenges. 

Because accountability reports are the most widely visible mechanisms for sharing successes 
and challenges for schools and systems, agencies must carefully consider the messages sent 
by accountability reports, and the desired responses from stakeholders. The choice of 
information placed in these reports publicly reflects the agency’s values. Even when legisla- 
tion dictates much of the content of the reports, as is the case in many jurisdictions across 
the country, report designers — the individuals who interpret and implement the reporting 
requirements — should plan reports and the materials that accompany them to suit local 
needs and interests. In doing so, they will contribute to the ultimate goal of an accountability 
system: improved educational achievement. 

To assist an SEA or LEA to create “a clear purpose” and achieve a “well-shaped message,” 
CCSSO recommends three critical ingredients in developing an accountability report: 

■ Define planning and implementation phases, 

H Use effectiveness as the guiding principal in the reporting system, and 

Engage in a report and preparation process that is inclusive of both context 



and goals. 
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Implementation Issues 

As this process is contemplated by SEAs and LEAs, it is important to think strategically in 
the short- and long-term when considering the two phases of reporting: planning/ 
preparation and implementation. As with any project, agencies should think differently 
about planning activities and implementation activities in terms of staffing and budgets. 
Planning/ preparation activities require levels of intensity and focus that probably cannot be 
sustained for very long. Implementation activities must be sustained indefinitely— and so 
should be staffed and budgeted accordingly. 

Use Effectiveness as the Guiding Principle of Reporting 

The recommendations for reporting found in ^ Guide to Effective Accountability 'Reporting are 
based on the principal of effectiveness: the degree to which the document conveys the 
intended message to the intended audiences and successfully encourages them to act on it. 
That may sound simple, but determining the message, the audience, and what the desired 
outcome requires a careful planning, above and beyond simple compliance with the law. 
Effective reports are meaningful and memorable — they carry a message that “sticks” with 
the stakeholder, provide solid answers to important questions, and spur stakeholders to take 
some kind of action — such as deciding to learn how to increase a child’s literacy skiUs or 
getting involved with a parents’ group at a school. In other words, an effective report is one 
that presents a coherent message, supports valid interpretations and communicates this 
message in a way that is accessible and useful for the right people in the right way at the 
right time. A list of some issues related to each of these concepts is in Figure 3 on pages 17 
and 1 8, and in Appendix B. 
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Steps in Report Preparation 

Regardless of the resources available to the agency — from staff to information 
management systems — there are some basic process elements to consider: 

1. Take stock of the current reporting system, 

2. Form the design team, 

3. Review other agencies’ reports, 

4. Design a dissemination plan, and 

5. Sketch out the reports. 

1 . Take Stock of the Current Reporting System 

As the first step in creating an accountability reporting system, agencies shoiald establish a 
reporting inventory that includes — to the extent possible — all reports that bear some 
relevance to public accountability reporting. The reporting inventory wiQ reveal what types 
of information are already published, by whom, and when, how, and to whom they are 
distributed. This will point out sources of accountability-related information and perhaps 
also where there exist inconsistencies, unintended overlaps, and holes in the reporting 
system. This inventory can help front-end planning, providing the opportunity to build or 
renew an efficient reporting system — that is, a system in which the reports meet the 
majority of intended needs with rninimum resources necessary to do so effectively. 

Assessment reports in particular are highly relevant to accountability reports and are an 
essential part of the reporting inventory. Too often, systems for reporting accountability 
information and systems for reporting assessment information are developed indepen- 
dendy. This will almost certainly end up confusing stakeholders: consider the difficiolties 
faced by parents trying to reconcile the school report card with the information in their 
child’s individual student report from the state assessment. As a result, it is a good idea for 
the content and style of the assessment reports to mesh with those of the accountability 
reports. 

Evaluation ^ 

In addition to these specific areas of consideration. It would be wise to build into 
the design process o few features that will allow report evaluation. These should 
focus on whether reports ore indeed effective (coherent valid, and accessible), 
Ideally, a periodic and formal evaluation of the reporting system would be 
regularly undertaken to ensure that the reports generated ore achieving the 
intended goals. Plus, any formal evaluation of the assessment system or 
accountability model should include the accompanying reports. 
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Finally, reports on teacher quality, dropout or graduation rates, promotion/ retention rates, 
enrollment, fiscal resource allocations, and several other areas are also highly relevant to the 
accountability reports because they include indicators that the accountability reports will 
direcdy use, break down, or summarize. 

It is important, also, to note the messages in all of these existing reports. The up— front 
planning required in creating an effective accountability reporting system provides the 
opportunity to shape the messages in all these reports to reflect the goals, objectives, and 
values of the community. 

2. Form the Design Team 

As discussed earlier, it is important to get as much input as possible on the evolving report- 
ing system. A systematic way of accomplishing this goal is to form a ""design team” for 
this purpose. This team should meet several times during the design and implementation 
periods to review plans and progress, and to provide recommendations and suggestions 
for the reports. Like many other committees that agencies work with, this team should not 
be in a position to give formal approval; rather, it should provide recommendations based 
on the members’ diverse range of expertise and experience. 

This design team should be made up of state and local agency staff and other types of 
interested stakeholders. The following are examples of personnel to consider for member- 
ship on this team: 

■ Tide I/Federal programs personnel; 

■ Reading and mathematics content specialists; 

■ Assessment directors; 

■ Accountability or evaluation analysts; 

■ Public Relations managers; 

■ Special education and ESL/bilingual specialists; 

■ Parents of students in Tide I schools; 

■ Education journalists; and 

■ Business leaders active in education. 

This committee wiU be most useful it if crosses jurisdictional boundaries. State and local 
agencies both produce many reports on the quality and effectiveness of their education 
systems and programs, and the ultimate responsibility for these vary by location: in some 
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states, districts have complete responsibility for designing and producing reports, such as 
school report cards. Elsewhere, states design and produce — or design and require districts 
to produce — such reports. Thus, while planning for the accountability reporting system, it is 
valuable to include as many varied voices as possible to comment on messages, design, and 
dissemination issues. ^ 

3. Review Other Agencies' Reports 

Chapter 2 of this guide offers examples of good practice for developing reporting systems; 
Appendix C includes resources and samples of good reports. Add to these models by 
gathering reports from other agencies, both in paper and web format. The result should 
provide both examples to imitate and those to avoid. There is no reason to reinvent the 
wheel for each report, either in content or format; in many cases, good reports have 
resolved problems that may be common across states and jurisdictions. Finally, consider 
contacting someone in charge of creating a useful report to learn the problems they encoun- 
tered and the solutions employed. This peer— to— peer interaction may provide a keen 
advantage to agencies moving forward with new or reinvented reporting plans. 

4. Design a Dissemination Plan 

Part of the challenge in designing an effective accountability system is making sure everyone 
gets the information they need. A fourth step in the up-front design process is developing a 
dissemination plan, literally a written document that lays out who gets which document and 
by what delivery means. 

At a minimum, a dissemination plan should include: 

■ Goals and objectives for the reports; 

■ A description of the target audiences for the reports; 

■ The intended use for the reports; 

■ The schedule for annual production and dissemination of the reports; 

a The evaluation plan for the dissemination system: do the documents reach the 
target audiences effectively and efficiendy? 

To be more effective, the dissemination plan should also include specifics on the distribution 
elements, not just about the report itself: 

a Guidance on printing and distribution of the reports: 

- For SEAs, will the reports be printed and sent directly to districts and 
schools, or wiQ districts and schools be required to print and distribute the 
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reports from electronic files? What sort of quality control will be in place? 

— How will the reports be packaged for distribution? Plain or specially 
designed envelopes? Will accompan)dng documents be in the same 
package? Will a bulk or other discounted rate need to be arranged? What 
sort of quality control will be in place? 

— Which stakeholder^ not associated with districts or schools will receive the 
reports? 

— How will staff handle incoming requests for reports? Will there be a 
charge for postage? Where will extra copies of the reports be stored? Will 
requests for previous years’ reports be honored? 

s Guidance on Internet distribution: 

— Will static versions of the reports be available, such as pdf versions of 
the paper reports? 

— Will interactive reports be available, ones that allow stakeholders to view 
indicators or graphics related to a specific question? 

° Will these indicators or graphics be ^^canned”, meaning that they 
are pre-produced and simply called up when a user selects from a 
set of options? 

or 

° Will these indicators or graphics be dynamic, meaning that analyses 
are conducted in real time in response to a user query? 

— Will links to district and school websites be included? 

— How will web reports and paper reports compare? 

— How will stakeholders find these reports? Through direct links or 
keyword searches? How visible will they be from the front page of the 
agency’s web site?. 

— Who will maintain the webpage? 



IT Outsourcing 

The complexity of the data and analyses systems increases dramatically across 
these three options (static, canned, dynamic). In the absence of a highly 
sophisticated IT department, SEAs and LEAs should consider contracting out for this 
work. A consultant could be hired only to design a system, or to build the data 
warehouse and set up the analysis and interface software. As an alternative to 
i producing custom systems, some states and districts are supplying their data to on- 
I line education reporting organizations. This is typically a free or inexpensive 
j approach, but reduces control over how data are reported publicly and are not 
! likely to meet all reporting goals and requirements. 
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— How often will data files be updated? Will the interface be reviewed and 
updated periodically as well? 

5. Sketch out the Reports 

Finally, based on the reports available, along with advice from advisory groups, agencies 
should make a rough sketch of what to put on the pages of the report. At a minimum, 
create a list of data elements that wUl be included. Even better would be to actually draw a 
rough picture of collective vision of the report. It should include some ideas on: 

■ How many pages the report wUl be and on what sized paper; 

■ The desired balance between text, tables, and graphics; and 

■ How the report will be organized so that it clearly conveys the message and 
presents information in order of priority or interest. 

Ongoing Reporting issues 

As the planning process proceeds, it is important to plan for the future of the accountability 
reporting system. Consider issues such as report storage and access, long-term distribution, 
updating the reports as new or better data becomes available, and staffing. Figure 3 on 
pages 17 and 18 provides a summary list of issues to consider for the implementation 
phase of accountability reporting. 
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Figure 3: Issues in Developing Effectve & Efficient 

Reporting Systems 

Effectiveness: Message, Content, Access 

Planning/Preparation Phase 

■ What message should the report convey? 

■ Who are the audiences for the report? 

■ What are the audiences meant to be able to do with the information in the 
report? . 

■ What indicators will best convey each component of the agency's message 
to these audiences? 

■ What are the separate components of the overall message and how will 
these components be represented in the report? 

■ Will the balance across report components parallel their conceptual 
balance in the intended message? 

■ What other information should be included in the report to facilitate 
appropriate interpretations? 

■ What other documents, such as evaluation or research reports, professional 
curricular or instructional guidelines, or agency policy statements, should 
be linked or referred to in the report to provide important context for its 
interpretation? 

■ Should any other document or reports produced by the agency be revised 
to support cohesion across the reporting system? 

Implementatjon Phase 

■ Is the report readily accessible to the target audiences? What sort of 
quality checks are in piace? 

■ Are agency staff trained and prepared to answer questions about the 
reports, via telephone, email, or letter? 

■ Have local educators been provided with adequate training and 
documentation on how to interpret and use the information in the reports? 

■ Have educators and community members been targeted to serve as 
'critical friends' by providing regular or on-demand feedback on the 
reports? 
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Efficiency: integration, Data Systems, Dissemination 
Pionning/Preparotion Phase 



■ What reports does the agency already produce that could meet new 
requirements, perhaps with some modification? 

■ Are the desired indicators already produced and disseminated eisewhere 
in the agency? Should any of the new indicators deveioped replace 
current indicators being used elsewhere? 

■ Do existing data coiiection instruments, cieoning processes, and timelines 
support the timeiy production of the accountability reports? 

■ If additional data must be collected, how con this be done without 
substantially increasing the burden of local educators and administrators 
(e.g., by replacing or revising items on existing surveys or by allowing those 
providing the data with options for easier filing, such os electronic 
templates for data submission)? 

■ Are the data management and analysis systems for various data elements 
integrated (or at least coordinated) so that the indicators ore readily 
available when needed for report publication? 

Implementation Phase r 



■ How is the report to be disseminated to parents? Other stakeholders? 

■ If a paper document is to be sent out, con this be coordinated with the 
dissemination of other documents, such os assessment reports or student 
report cords? 

■ Con the report cords, or information about them, be distributed to the 
community via a statewide or local newspaper? 

■ Con members of the broadcast medio mention the report cords and 
where to find them on the air? 

■ Has a system been developed to accept input from stakeholders on the 
quality of the reports and the dissemination system? How will this input be 
used to improve reporting practices? 

■ What other information will stakeholders need to be able to understand 
and use the report cord and is this information readily accessible to them? 
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Chapter 2: Designing Reports for the 
No Child Left Behind Act 



Chapter 1 focused on establishing the principles and identifying the requirements that should 
drive an agency’s report development processes. This chapter focuses on specific content 
requirements of NCLB accountability reports. 

NCLB requires a specific number of indicators, and information to be included in report 
cards, some of which are new indicators for states. Most require more disaggregation than 
currently required by states. These fall into three basic categories: 

■ Assessment reporting, 

■ AYP reporting, and 

■ Indicators of teacher quality, school conditions, and outcomes. 

The specific requirements within these categories are summarized in Figure 4. The pages 
that follow contain examples of how to report the required information in each category in 
ways that are both effective and efficient. 
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Figure 4: NCLB Report Card Reporting Requirements 

Under the NCLB Act States and school districts (LEAs) are required, not iater than the beginning of the 2002-03 schooi year, to 
prepare and disseminate annuai report cards. Requirements for the data eiements to be reported are set forth under section 
1 1 1 1 (h) of the iaw and summarized in this figure. Aithough schoois are not required to prepare and disseminate their own 
reports, information about whether a schooi has been identified for improvement and how a schooi's students are achieving 
must be included in the State and schooi district reports. 

States must aiso report annualiy, beginning with school year 2002-03, to the Secretary of Education information on their 
progress reiated to requirements set forth in section 11 11 (h)(4). 

States and schooi districts reports are required to be concise and presented in "an understandable and uniform format and, to 
the extent practicable, provided In a ianguage that parents can understand" (sections 1 1 1 l(h)(l)(B)(ii) and 1111(h)(2)(E)). 
Additionaiiy, schooi district reports must be made "wideiy avaiiabie through pubiic means such as posting on the internet, 
distribution to the media..." (section 1 1 1 1(h)(2)(E)). 



Items to be Reported 


state 

Report 

Card 

mih(l)C 


LEA 

Report 

Cad 

nilh(2) 


School 

Report 

Card 

mihf21B 


Annual 
Report 
to Sec. 
mihrdi 


Assessment Information 


For each grade and subject tested: (1) percentage of 
student assessed and (2) information on 
students at each state defined proficiency 
levei, in the aggregate and disaggregated bv: 


X 


X 

Compared 
to state 


X 

Compared 
to state and 
district 


j 

X 


Gender^ 


X 


X 


X 




Major Raciai and Ethnic Group 


X 


X 


X 


X 


Engiish Proficiency Status 


X 


X 


X 


X 


Migrant Status^ 


X 


X 


X 




Students with Disabiiities 


X 


X 


X 


X 


Compared to Non-disabied students 


X 


X 


X 


X 


Economicaily Disadvantaged 


X 


x“ 


X 


X 


Compared to Non-Economicaiiy Disadvantaged 


X 


X 


X 


X 


Most recent 2 year trend in each subject and grade ievei 
in student achievement^ 


X 


X 


X 


X 


State's progress in developing required assessments 








X 


Accountability Information \ | 


comparison oetween me actual achievement for each 


X 


X 


X 





group^'** and the State's annual objectives for AYR: 
Ali Students 

Economically Disadvantaged 

Major Raciai and Ethnic groups 

Students with Disabiiities 

Limited English Proficiency Students 



Aggregate and disaggregate^ information on other 
academic indicators that the State has selected for AYP: 
Including graduation rate for secondary and an 
academic indicator for Elementary, Middle schools 


X 


X 


X 


Aggregate information on any additional indicators the 
state may use to determine AYP 


X 


X 


X 


Performance of LEAs regarding achieving AYP 


X 


X 




Number of Schools identified for Improvement 


X 


X 


X 


Names of Schools in improvement 


X 


X 


X 


Percentage of Schools identified for improvement 




X 




How long the Schools have been identified for 
improvement 




X 




Whether school has been identified for improvement 






X 


Reason(s) school was Identified for improvement 






X X 


Measures taken to address achievement problems of 
Schools identified for improvement 






X 
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Items to be Reported 


state 

Report 

Card 

lllhmc 


LEA 

Report 

Card 

llllh(2) 


School 

Report 

Card 

1111(2)6 


Anriual 
Report 
to Sec. 
llllh(4) 


Teacher Information^ 

Professional qualifications of teachers as defined by 


X 


X 


X 


X 


the state 











Percentage of teachers teaching with emergency or X X x 

provisional credentials 

Percentage of classes taught by highly qualified ^ 

teachers in the state, LEA, and school 

Percentage of classes In the state not taught by highly XXX 

qualified teachers (aggregate and In the highest and 
towest quartile schools based on poverty) 

English Language Proficiency 

Information on the acquisition of English proficiency X 

by LEP students 

Supplemental Services and School Choice 

NuTT^r of students and schools that participated in ^ 

supplemental services and public school choice 

Time Line — ■ — 

beginning of the Beginning 

2002-2003 SY with school 

year 

2002-2003 

’Except thof such disaggregation shall not be required in a case in which the number of students in a category is insufficient to 
yield statistically reliable Information or the results would reveal personally Identifiable Information about an individual student 

^Not required for Adequate Yearly Progress 

^In any year before the State begins to provide information described above, information on the results of student academic 

assessments 

^Including students who are enrolled i n a school or LEA for a full academic year, as defined by the state. 

®Sec. 1119 a/b requires reports on schools, LEAs, and the state— regarding meeting measurable objectives related to the 
percentage of highly qualified teachers in core academic subjects: English, reading or language arts, mathematics, science, 

foreign languages, civics and government, economics, arts, history, and geography . 

Major racial and ethnic subgroups: African American/Black, Asian/Pacific Islander, American Indian/Alaskan Native, Hispanic, 
White 



, Guide to Effective Accountability Reporting 

cKJC 



21 



26 



Assessment Indicators 
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In terms of the number of indicators involved, the majority of the information that NCLB 
requires states and districts to report involves performance on the state or local standards- 
based assessments. On the school, district, and state level report cards, NCLB requires 
reporting of total and disaggregated performance information: 

■ Student acliievement by academic performance level, 

■ Most recent 2-year trend in each subject and grade for All Students, 

■ Comparison between the actual achievement and the State’s annual objectives 
for AYP, and 

■ Percent of students assessed. 

Clearly, the amount of information here is substantial, especially since the assessment data 
must be disaggregated by subgroups, subject, and grade: 

■ School/district student population, and 

■ Each specified subgroup 

— Racial/ ethnic groups — ^including at least 
° African American/black, 

° Hispanic, 

° White, 

° Asian/Pacific Islander, and 
° American Indian /Alaskan Native; 

— Male and Female; 

- Students with disabilities AND students without disabilities; 

— Limited English Proficient students; 

— Migrant students; 

— Economically Disadvantaged AND Non— Economically Disadvantaged 
students. 

The simplest way to present a large amount of information like this is to use a series of 
tables. The major organizing variables should be grade level and test. That is, decide whether 
to report on all tests within a grade level or report for aU grade levels within a test. 

The sample in Figure 5 on the next page illustrates how this could look. In this example, aU 
required assessment indicators are presented for a single grade. The proportion of students 
tested and the percent scoring at each performance level for the school are presented for the 
current year, along with the school’s grade-level trend across two years for the ‘‘aU students” 
category. 
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Figure 5: Sample Assessments Report for a School, Single 
Grade Level, Language Arts Assessment 



Grade 3 Language Arts Assessment 
Student Group 


% tested 


% at eoch proficiency level 
Level 1 Level ll Level III 


Level IV 


All students 


School 

District 
. State 


2001-02 

2000-01 

2001-02 

2001-02 


95 

94 
93 

95 


10 

11 

15 

18 


20 

21 

23 

22 


29 

23 

35 

36 


41 

45 

39 

24 


Female 


School 


2001-02 


96 


9 


12 


34 


55 




District 


2001-02 


94 


8 


12 


31 


58 




State 


2001-02 


95 


8 


14 


30 


48 


Male 


School 


2001-02 


92 


15 


23 


32 


30 




District 


2001-02 


95 


17 


32 


35 


40 




State 


2Q01^Q2__ 


97 


14 


23 


36 


28 


African American 


School 


2001-02 


94 


15 


20 


31 


36 




District 


2001-02 


96 


9 


12 


31 


31 




State 


2001-02 


93 


14 


25 


35 


25 


Hispanic 


School 


2001-02 


85 


11 


10 


34 


26 




District 


2001-02 


84 


10 


14 


32 


37 




State 


2001-02 


84 


12 


12 


31 


45 


Asian/Pacific Islander School 


2001-02 


- 


- 


- 


- 


- 




District 


2001-02 


< 


< 


20 


38 


34 




State 


2001-02 


92 


9 


20 


35 


34 


American Indian/ 


School 


2001-02 


- 


- 


- 


- 


- 


Native Alskan 


District 


2001-02 


- 


- 


- 


- 


- 




State 


2001-02 


92 


4 


21 


38 


41 


White 


School 


2001-02 


94 


5 


16 


34 


25 




District 


2001-02 


95 


15 


15 


34 


36 




State 


2001-02 


95 


5 


20 


35 


41 


Students with 


School 


2001-02 


95 


18 


21 


39 


47 


Disabilities 


District 


2001-02 


96 


17 


25 


34 


16 




State 


2001-02 


91 


19 


32 


31 


14 


Students without 


School 


2001-02 


98 


8 


25 


38 


39 


Disabilities 


District 


2001-02 


94 


14 


20 


37 


18 




State 


2001-02 


95 


16 


41 


3] 


15 


Migrant Students 


School 


2001-02 


85 


18 


38 


36 


8 




District 


2001-02 


86 


14 


20 


35 


54 




State 


2001-02 


84 


11 


32 


29 


54 


Limited English 


School 


2001-02 


79 


1 1 


20 


34 


52 


Proficient Students 


District 


2001-02 


85 


13 


10 


’ 30 


24 




State 


2001-02 


85 


12 


16 


29 


24 


Economically 


School 


2001-02 


91 


10 


12 


30 


23 


Disadvantaged 


District 


2001-02 


95 


8 


12 


35 


45 




State 


2001-02 


94 


10 


14 


34 


42 


Non-Economically 


School 


2001-02 


98 


8 


10 


38 


41 


Disadvantaged 


District 


2001-02 


95 


7 


18 


41 


44 




Stale 


2001-02 


94 


9 


_L5 


23 


43 



< Group below state definition for statistically reliable results. 
- No students in group. 
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Although this is a great deal of information to put into one table, several key design 
features do help: 

■ Grid lines have been deleted in some cases and lightened in others to reduce the 
amount of interference with the text, 

“ The most important information to most readers is larger. This helps the 
reader to quickly skim the table and see patterns; 

— The symbols have been eliminated within the table’s data cells and 

numbers have been rounded to the nearest whole. Results may be reported to the 
tenth decimal place, but additional detail tends to make a table more difficult to 
read; and 

— All words are speUed out in full so the reader doesn’t have to refer to a separate 
key. 

This sample table reports the percent of smdents scoring in each performance level. The 
number of non-participating students is not explicidy reported in this table. However, the 
aim was accomplished by reporting the percentage of students who did participate — an 
indicator for which there is an explicit goal and requirement. 

’ The darker gridlines separate the sets of subgroups so that each student would be repre- 
sented only once within the rows between the darker horizontal lines. This, too, helps the 
reader to focus on reasonable comparisons. In this table, multiple years of data are kept 
together within grade level; this supports readers’ reasonable comparisons of performance 
across time within a grade. 

This text accompanying this table should be minimal, but used to describe the assessments 
and definitions of the subgroups. It should also include some cautions to readers about 
making comparisons across groups, even from year to year, and against inferring that 
relatively small differences are meaningful or that differences are the result of innate abilities 
or a biased test. 
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Adequate Yearly Progress Indicators 

At the school, district and state level report cards, the NCLB Act requires reporting the 
aggregate and disaggregated performance and academic indicator information: 

■ Comparisons between the actual achievement of students and the State’s annual 
objectives on academic assessments; 

■ Aggregate and disaggregated (not required in AYP calculation) information on the 
other academic indicator used for adequate yearly progress; 

■ Whether the school has been identified for improvement (including corrective 
action and restructuring). 

In the case of the school level report, NCLB requires a comparison between the achieve- 
ment of students in the school on academic assessments and other indicators of AYP 
compared to students in the district and state as a whole. The sample in Figure 6 illustrates 
how such a report could look at the school level. This table includes indicators that are 
reasonable to compare — school, district, and state levels of performance — right next to 
each other. Since NCLB requires the reporting of all of this information, an alternative is to 
provide a single summary table, which lists only the information for each level of statistic. 

Figure 6: Sample School Level Accountobilily Report Cord: 2002-2003 
School: Thoreou Elementary Grade Span: K-5 

’ ^ ■ Academic 



Student Group J Language Arts Mathematics IndicatoKs)* 





% tested 
Goal: 95% 
School District 


State 


% Proficient 
Goal :50% 
School District 


State 


School 


% tested 
Goal: 95% 
District State 


% proficient 
Goal: 40% 
School District 


State 


Attendance 
Goal: 90% 
School District State 


All students 


96 


94 


95 


49 


51 


50 


96 


94 


94 


39 


39 


41 


89 


88 


91 


African American 


95 


95 


93 


54 


54 


52 


95 


94 


96 


40 


40 


41 


90 


88 


91 


Hispanic 


94 


93 


84 


< 


< 


51 


94 


93 


92 


< 


< 


41 


88 


88 


89 


Asian/Pacific Isl. 


_ 


94 


95 


— 


51 


52 


- 


94 


95 


- 


< 


54 


- 


< 


94 


American Indian/ 
Native Alskan 


- 


< 


96 


- 


< 


58 


— 


< 


58 


— 


< 


96 


— 


< 


96 


White 


96 


92 


94 


55 


55 


56 


95 


95 


96 


41 


38 


40 


87 


90 


96 


Students with 
Disabilities 


97 


99 


95 


< 


< 


41 


94 


98, 


98 


< 


< 


34 


84 


85 


85 


Limited English 
Proficient Students 


92 


95 


79 


39 


41 


40 


92 


98 


97 


24 


45 


31 


90 


91 


89 


Economically 
Disadvantaged 


95 


96 


91 


42 


38 


40 


96 


97 


98 


42 


40 


38 


91 


95 


94 



This school has not met the Adequate Yearly Progress (AYP) requirements as defined by the state for the 2002-03 school 

year. This is the first year the school has not met the AYP requirements under the Fed eral No Child Left Behind Act, 

* This can include graduation rate for secondary schools or other academic indicator for elementary and middle schools, as well as 
optional indicator(s) as determined by the State. 

< Group below state definition for statistically reliable results. 

~ No students in group. 

##: Reported, but not included in AYP calculation 
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An alternative could be to report the percent scoring at or above proficient and the percent 
scoring below the lowest cut score. This would help the reader to understand a bit more 
about the score distribution. When making these decisions, keep in mind that at some point 
more information will become too much information. 

Other Academic Indicators 

NCLB requires for AYP reporting one additional indicator beyond the assessment indica- 
tors for determining progress of each school and district. Graduation rate is required for 
high schools, but for other grades, the indicator is selected by the.state and are a subset of 
those that most agencies already report. They include outcomes — such as graduation rate — 
and other variables that may be related to outcomes such as class size, attendance rates, and 
levels of school violence. However, SEAs and LEAs should be careful to ensure that the 
local defimtion for these indicators matches the NCLB ones. For example, the graduation 
rate defimtion for NCLB is different from the way in which many states currently calculate 
it. (States and districts are welcome to use additional indicators beyond graduation rate for 
high schools and another indicator for other grades, as desired, but these must be included 
in AYP reporting.) 

The other academic indicators are not included in the annotated list in Appendix A because 
NCLB does not specify how to calculate these, even though it requires at least one be used 
in the formula for determining AYP. But agencies must include any indicator in the report, 
whether it appears on the list of optional ones in NCLB or not, if it is used to determine 
AYP. These indicators should be calculated and reported in the manner that best reflects 
the agency’s systems and values. 

Reporting Teacher Indicators and School Demographics 

A second type of indicator used in accountability reports reflects characteristics of schools 
that do not stem from performance or participation on an academic test. In a broad sense, 
this category includes a number of indicators that have been a part of state and local 
indicator reports for years, including student demographics, quality of the teaching staff, 
school programs, characteristics, resources, and finance indicators. However, several types 
of indicators are specified by NCLB requirements. 

iGaChGr Indicators The NCLB Act specifically requires teacher information to be in- 
cluded in the state, district and school report cards. The professional qualifications of 
teachers, percentage of highly qualified teachers in the aggregate and disaggregated by the 
poverty level of schools, and the percent of teachers with emergency or provisional 
licensure are all required reporting elements. An example of how teacher indicators could 
be reported is illustrated in Figure 7. Again, the more contextual information that can be 
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given, the more relevance these numbers will have to the reporting audience. At the same 
time, it is important to make sure the message is not lost in tables upon tables of data. 



Figure 7: Teacher Information relating to NCLB core academic 
subject areas: 



Professional Qualifications** 


Must meet state licensure standards, major in 
subject area tauqht 


% of teachers with emergency/ 


12% 


provisional certification 




% of classes not taught by Highly 15% 


Qualified Teachers 




in High Poverty Schools 


22% 


in Low Poverty Schools 


11% 



** As defined by the state. 



Highly Qualified Teachers as Defined by the NCLB Act When used with respect to 
any public elementary school or secondary school teacher, refers to a teacher that: 

■ Has obtained full state cerdficadon, which may include cerdficadon by 
alternadve routes 



or 

■ Has passed the state licensure examinadon and holds a license to teach in 
the state; and 

■ Has not had cerdficadon waived on an emergency, temporary or 
provisional basis 

The NCLB Act contains specific requirements by grade taught, including: 

A New Elementary School Teacher: 

Must possess a Bachelor’s degree and pass a “rigorous” state test of subject matter knowl- 
edge and pedagogy in reading, writing, mathemadcs, and other subjects of the elementary 
school curriculum. 

A New Middle or Secondary School Teacher: 

Must possess a Bachelor’s degree and pass a “rigorous” state test in each of the academic 
subjects in which the teacher teaches 
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or 



Successfully completes in each academic subject taught: 

— An undergraduate major; 

~ Graduate Degree; 

— Coursework equivalent to an undergraduate major; or 

— Advanced certification or credentials 

^lementaryy M.iddky or Secondary Teacher who is not new to the profession: 

Must possess a Bachelor’s degree and meet the applicable requirements for 
teachers new to the profession; 

or 

Based on a high, objective, uniform State standards of evaluation 
demonstrates competency in each academic subject in which the teacher 
teaches. 

See Appendix A for more detail. 

^Highly Qualified Teacher applies to all '‘core academic subjects”: English, reading or 
language arts, mathematics, science, foreign languages, civic, government, economics, arts, 
history, and geography. 

School Demographics 

The biggest challenge in reporting other indicators for schools, such as student and teacher 
demographic data, is in deciding how to organize them and where to find space for them 
given the demands of AYP reporting. Readers may want to see these indicators in two 
different places within a report card, (a) with other substantively related information, and 
(b) with indicators used to calculate AYP. For example, many agencies organize their report 
cards into sections such as basic characteristics of the student body (e.g., enrollment, pro- 
portions in race/ ethnicity categories, etc.); assessment performance; staffing (e.g., number, 
qualifications, absenteeism, etc.); school culture (e.g., attendance, class size, promotion/ 
retention, violence, etc.); and fiscal resources. 

Figure 6 represents the explicit requirements of the NCLB Act. However, a simple 
statement on whether the school has been identified for improvement should be expanded 
upon. Listing out the categories where AYP was met and not met, providing information 
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on how a school has made progress in its lowest achieving subgroups, and presenting a plan 
for the school to improve all provide the contextual information a parent would need to 
understand the designation identified for improvement. A discussion on reporting the AYP 
results is located in the last section of this chapter. 

Summarizing Accountability Information for a Schooi or District Report 
Card 

This chapter has focused on specific content requirements of the NCLB Act accountability 
reports. The sample report pages and description of indicators defined by NCLB law and 
regulations highlighted the information needed to determine a school or district’s status. 
However, another key concern in designing accountability report cards is how to present 
summary information about the status of a school or district to make it easily reported and 
understood by parents and the public. 

From the perspective of the public, the key question in relation to accountability with 
NCLB is the summary judgment about their school, or district, i.e. Does it ‘‘make the 
cut” — ^Adequate Yearly Progress (AYP)? or Does it not make annual AYP? Is it “identified 
for improvement”? If so, Whatis being done to help? The critical summary information 
about a school or district can be organized in three categories: 

1. Outcome 

■ Yes (made AYP) vs. No (did not) for prior school year 

■ Number of consecutive years not made AYP 

2. Reason 

■ Number of indicators met vs. Number not met for the school year 
reported 

■ List of indicators Not met 

3. Consequences 

If AYP is not met, whether the school is eligible for Public School Choice or 
School Improvement (including corrective action, restructuring, etc.) 

e.g.. Choice: Students in this school may choose to transfer to another public 
school in the district. 

Because this summary information is paramount for many stakeholders, one reporting 
strategy is to place the summary results on the front page of the report card. This allows a 
parent, lay person, or educator to quickly gain an overall picture of where the school or 
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district stands in relation to accountability requirements under NCLB, as well as gauging 
how well/ poorly the school or district is perforrning across the indicators, and what will 
happen during the coming school year. 

For summary information, such as the names of schools within a district that are in need of 
improvement (including correcting action, restructuring, etc.) and the length of time that 
these schools have had that status, a simple table will work best. Smaller agencies could list 
all schools and indicate AYP information for each one. Larger agencies may be better 
served by a summary with only identified schools /LEAs. 

Finally, to provide a positive, action-orientation to the report card format, one additional 
summary category should be included on the cover of the report or close to the front: 

4. Actions Toward Improvement 

Primary resources and programs aimed toward gaining AYP 

In the initial phases of developing report cards, it may not be possible to simultaneously 
report the indicators and summary results from the prior school year as weU as the all the 
various programmatic responses planned by the school or district. However, it should be 
possible to incorporate in the report card highlights which major categories of state or 
district funding will be available to assist the school/ district and which programmatic 
initiatives will be applied in the coming school year to address the areas of low 
performance. This additional category will help move accountability reporting in the 
direction of communication and problem-solving rather than simply statistics. 

The four summary reporting categories about the school or district should be followed by 
a table of contents which relates the data for each separate indicator to the summary results, 
and helps guide the reader to subsequent sections of the report card for further detail. 
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Chapter 3: How to Design 

As the process for communicating the information required by NCLB is developed, 
whether in a paper report, on the web, or both, specific universal design principles should 
be noted. Good report design represents clear thinking on the part of its author(s). An 
effective accountability report is: 

■ Easy to read and clearly states a well-defined message that stakeholders can 
understand and use; 

■ Accessible to the target audiences, both physically and linguistically; 

■ Accompanied by adequate interpretive information; 

■ Supported by evidence that the indicators, other information, and suggested 
interpretations are valid; 

■ Coordinated with other reports within the reporting system: 

— Across paper and electronic versions of report cards, and 
— Across reports cards and assessment reports. 

Effective reports are clear, easy to understand, and memorable. Every mark on the page 
should be as small but as clear as possible. In other words, minimize unnecessary ink on the 
page — ^words, numbers, pictures, graphics, lines — and allow less necessary elements — alines 
on tables for example — to recede into the background by making them as pale as possible. 
Finally, as discussed, in addition to making sure report is cohesive in terms of both sub- 
stance and message with other reports. It also helps if they have a similar “look” in terms 
of graphics, color, and overall design. 

This discussion is organized into three sections: 

■ Crafting the language so that it is clear and easily understood; 

■ Using graphics to highlight important information without cluttering the page 
or distracting the reader; and 

■ Aligning the report cards with other documents, such as Spanish-language 
versions of the report card or interpretive information about the assessment 
system. 

The overarching message across all of these sections is this: make sure these reports are 
assessable to the average person. While designing the reports, ask, '"Would my neighbor 
understand this?” Make sure the answer is yes. 
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Crafting the Language 

Once the overall messages for the report card have been determined, but before the text is 
written, it is important to think about the audience. Certainly, with a wide variety of stake- 
holders who will be motivated to look at a report card, the document must contain a wide 
range of information about a school, a district, or the state that is presented in a clear and 
easy to read format, free of jargon (both verbal and numerical), and points to sources of 
other relevant information. And, it is important to keep in mind that the target audiences 
represent a wide range of linguistic diversity, technical sophistication, access to and comfort 
with technology, and motivation to seek out and act on information about their schools. 
This means that the language must be crafted so that parents with no experience with 
educational terminology or evaluation concepts can understand the message, while at the 
same time it rings true to seasoned professional educators. 

Consider: 

■ The language used within a report card should be jargon-free. Do not use: 

— Lingo, like ''disaggregated”, "standards-based test”, "alternate assessment”, 
"performance index”, or any idiomatic words or terms, without providing a clear, 
brief definition; 

— Acronyms, Uke "FARMS”, "AYP”, "FTE”, "CRT”, or any variable abbreviations, 
without first spelling them; 

— Statistical or psychometric terms, such as "chi-square”, "p-value”, "theta”, or 
"coefficient”, unless absolutely necessary — the best way to demonstrate technical 
sophistication is by explaining a complex concept simply and elegandy to an 
unsophisticated reader; and 

— Statistics that are unnecessary to the message. 

■ The report card might not include much non-indicator text, especially since some 
indicators may be narrative. However, whatever text is provided should be easy to 
read and understand: 

— Use a combination of sentence lengths and avoid long and/or overly-punctuated 
sentences; 

— Keep paragraphs short. At most, five sentences or two inches long; 

— Use an active voice rather than a passive one; 

— Surround text by white space; and 
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—Choose a font that is easy to 
read: 

• avoid boldface except in 
headings and to highlight specific 
words or phrases, 

• don’t combine font styles, such 
as boldface and italics, except 
perhaps for some headings, 

® choose fonts that are easy to 
read, and that reproduce well, 

® use as few fonts as possible, and 

® carefully use color to illustrate 
points rather than to distract from 
them. 

— Subject the text to multiple reviews: 

• Use the design team to review the text for readability and clarity; and 

• Have teachers and parents connected with a diverse range of schools read the 
text and provide feedback. 



try this 

Which of the lines below is easiest to read? 

Use a font that Is easy to read and doesn't distract or confuse. 
Think about how numbers, such as 1, 3, 5, 7, and 8, appear In 

the text. -n. - 1 u 

Use a font chat is easy to read and doesn’t distract or confuse. Think about 
how numbers, such as 1, 3, 5, 7, and 8, appear in the text, 

USE A FONT THAT IS EASY TO READ AND DOESN’T DISTRACT 
OR CONFUSE. THINK ABOUT HOW NUMBERS, SUCH AS 1, 3, 5, 7, 
and 8, APPEAR IN THE TEXT. ^ 

orcoYvfUie/. ThCnJo about how ruAmhovy, sMcho4rl, 3, 

I 5, 7, a^nd/8, rtpp&cur Ivuth^te/xt. 



Using Graphics 

Because NCLB requires such a large amount of information to be conveyed, it is important 
to carefully choose how to display data elements. When used well, graphics can be a striking 
tool to illustrate messages in the report. The decision on how to present the data is the 
difference between a report that complies with the law, and one that shares a message about 
student achievement across groups and schools. Among the most common graphic forms 
are simple bar charts or histograms, which are useful for illustrating either the frequency of a 
characteristic across groups (e.g., number or proportion of students in each racial/ ethnic 
category or the proportion of students scoring in each performance level) or a trend over 
time (e.g., a school’s mean reading score for each of five years). 

For example: 

When deciding whether to use a graph or a table, consider whether the space and ink used 
in the picture (Like Figure 8) is necessary compared to the amount of information that could 
be conveyed in the same space in a table (like Figure 9). 
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Figure 9 



Figure 8 
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67 
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Building on this idea, consider 
combinations of tables and pic- 
tures, such as in Figure 10. In 
about the same amount of space 
used for the bar chart (Figure 8), 
Figure 1 0 graphically indicates the 
proportion of students scoring at 
or above the proficient level for 
each of five years and also pro- 
vides numerically the proportion 
of students in each of the three performance levels. Further, the simple horizontal line to 
represent the total scale with the tick mark to indicate the proportion above and below the 
proficient cut score allows quick comparisons between performance and the goal (100% 
proficient and above) as well as across years. The black subline to reflects the confidence 
interval around each score. 

This example shows how a few modifications in the approach to representing data, both 
numerically and graphically, can make a big difference in how much information can be 
provided in a limited space and how easy it will be for stakeholders to see what’s important. 

Some general rules : 

■ Graphical displays should reflect the critical elements of the message and on 
what readers should focus. As such, they should be used sparingly and 
strategically. 

■ Graphs are good for reporting trends, but not for conveying the actual 
numbers behind them. 



Figure 10 
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1997 
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44 


18 
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■ The more complex a concept is, the more likely a good graph can help convey 
it 

— If a series of numbers can be described in one sentence without any 
qualifiers, don’t graph these numbers. 

— Change over time is more interesting than status. Changes in proportion 
across groups over time are more interesting than change over time. 
Changes in proportion across groups over time in relation to a common 
standard are more interesting than changes in proportion across groups 
over time. 

How to Graph 

Once the decision is made on what to graph, it is necessary to begin to think about how to 

graph: 

■ Graphic displays do not have to take up a whole page, or even a significant portion of 
a page. In fact, bigger is usually not better. Rather than using a large proportion of the 
page with a graph, consider using a series of smaller graphs to highlight comparisons 
across time or groups. 

■ Do not accept the limits of graphic programs. If it is the best way to make the point, 
and someone can draw it, it can be programmed. 

■ Completely integrate graphics with words and numbers on the page. Embed them 
within the right context rather than in an appendix or on a separate page. 

— If the graphic might be useful for a stakeholder as a slide in a presentation, offer it 
electronically or create a separate toolkit with enlarged versions of graphics for this 
purpose. 

■ Create a stylebook which defines how each graphic should look: specify font styles and 
sizes, use of color and pattern, etc. For best results, create templates for each graphic 
and distribute these templates electronically to anyone who may need to produce 
graphics for these reports. 

The following four examples help illustrate these points. 
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Example A 



Earlier ia this section, a graphic illustrated these 
scores, the proportion of students scoring at or 
above the proficient level on the test indicated for 
each of five years. Figure 1 1 is an example of how 
not to graph these numbers. 

Clearly this graphic does not tell the story. In fact, 
it’s difficult to understand anything in this graph. 
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Consider the following: 



There is litde reason to use 3-D (three dimensional) graphics in the two-dimensional 
space of paper. Flqure 1 1 



• There is so much informa- 
tion in this graphic that some 
of the columns are hidden. 

• The background shading 
does not add anything to the 
graphic. Use as litde color as 
necessary, and none on the 
background of graphics, if 
possible. 

• The gridlines are distracting. 
If they must be used, do so 
sparingly and use as litde 
color as necessary to show 
them. 

• The legend is redundant. As 
part of the goal to integrate 
words, numbers, and images, 
try to just name columns, 
bars, or lines direcdy. 
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□ Math 
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An alternate way to handle this type of complex data is to use a table. In figure 1 2, notice 
that the gridlines are not black — the horizontal lines have been lightened to reduce distrac- 
tion from the numbers — and some vertical lines have been deleted altogether. This table 
shows the proportion scoring above and below the proficient cut point. However, to save 
even more space, consider only listing one of these columns, since they should always add 
to 100. An alternative would be to list the proportions at or above proficient and the 
proportion scoring in the lowest level, if this level is not adjacent to proficient. 



Figure 12 
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Or, in order to highlight changes in proportion over time, a table can contain small graphics 
similar to the first example in this section. In Figure 1 3, a simple gray line to reflect the total 
(100%) and a black block to represent the proportion of smdeints at or above proficient. By 
flipping the table, it is easy to make comparisons across years by running the eye down the 
column for each content area. The reader can also make comparisons across content areas, 
though not as easily. 

Figure 13 
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Example B 

If planning to use a pie chart, make sure this is the best option to present the data, and also 
make sure the graphic is easily readable. In the example below, the large number of sections 
results in visual confusion with multiple arrows and labels. Consider using another chart, 
such as the bar chart in Figure 15. The pie chart might better illustrate indicators with fewer 
elements. 



Figure 14 



Figure 15 




African American 
Hispanic 
White 

Asian/Pacific Islander 

American Indian/ 
Alaskan Native 

Other 



□ 



If a bar chart or histogram is chosen to illustrate the data, 

consider the following: 

• As fuUy as possible, integrate the words and numbers 
with the image. 

• Get rid of the unnecessary characters, such as percent 
signs. Just label the chart as one representing proportions. 

• Use as few ticks as possible to break up the scale. 25, 50, 
75 may be enough; maybe just 50. 

• Use as litde color, pattern, and ink as possible to illustrate 
the point. 

• Consider reducing the width of the bars. This helps to see 
the comparison and save space. It may be necessary to 
use fewer labels on the x-axis to fit with the reduced 
amount of space. 
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• Make the chart wider than it is tall. Both Figures 1 6 and 1 7 are proportioned this way — the 
one on the bottom is about as narrow as it could get without appearing out of balance. 

Notice how Figure 17 uses less space, ink, and color, yet still is able to convey the message. 



Example D 



A big design decision for accountability reports has to do with the use of color. When used 
strategically, color can enhance a report by drawing attention to critical information. However, 
caution should be used when planning whether and how much color to employ: 

a First, color can be distracting. In Figures 18 and 19, the two column charts are identical 
except for the color of the columns. In Figure 18, each column is a different shade; in 
Figure 19, aU columns are one color. Further, when colors in charts are very bright, they 
become distracting, making it - . _ ^ 

difficult to discern the ^'9^^ 18 

differences in the height of 
the bars. 80 



Even with categorical data 
that might be clumped 
together — such as scores for 
each of five schools for each 
of four years — it is not 
necessary to use a different 
color for each school. As long 
as the number of schools 
represented in the chart small, 
order of appearance is 
enough for a reader to be 
able to identify the pattern. 

Of course, an even easier 
alternate might be to present 
the four years of data 
grouped for each school. 



50 



20 



1997 



1998 



1999 



2000 



2001 



Figure 19 
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■ Second, die possibility exists that people might make photocopies of the report, or 
print a web report without a color printer, so it is important to color the graphic in 
such a way that the distinction will be clear in a black/ white copy. Also, each additional 
color usually increases the print costs of the document. 

■ Third, many popular colors — ^including red — are difficult for some readers to see 
clearly. Thus, color can, again, reduce readers’ accessibility to the report’s information. 

If using color in the reports, consider the following: 

■ Consider designing the document using only one color besides black. Shades of these 
two colors provides a wide range of possible combinations for graphs and accents. 

■ Especially since the report could be photocopied to black/ white, consider using white 
or off-white paper for the background of printed reports. It provides a strong contrast 
for text and graphics, and will not discolor when copied. 

■ Test printed reports by photocopying them on copiers of varying quality. Make sure 
color, text, and graphics transfer to copies in a way that doesn’t affect the amount or 
quality of the information the report is meant to convey. 

■ Test web reports by printing them out on printers of varying qualities — ^including some 
that are color and some that are not. 
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Conclusion: Aligning Report Cards With Other Documents 

Report cards are both stand-alone documents and part of a larger system for providing 
information to stakeholders. As stand-alone documents, report cards should present a full 
story by themselves. Stakeholders — especially parents and the public — should not have to 
search for essential information. Paper reports should be self-contained and accompanied 
by necessary interpretive information; web-based reports should include links to both extra 
information (e.g., brief definitions, prompts, etc.) and other documents. 

In planning the accountability reporting system, consider the following : 

■ Report cards must be accompanied or supplemented by other documents that support 
their use. For example: 

— Non— English— speaking parents and stakeholders must have access to the same 

information as English-speakers. Depending on local needs, it may be necessary to 
produce versions the report cards in other languages, provide supplements, inserts, 
or links for the report card in other languages, or support local translations of 
some report card information; 

— Directions to additional information should be embedded within the report cards; 
and 

— Professional development materials for educators should include information 
about the report cards and how to interpret them. 

■ The report cards must be aligned with assessment reports. This is not an additional 
burden; in fact, good alignment between the accountability and assessment reporting 
systems wiU save each time and trouble. 

— Accountability reports should refer to the assessment reports and to the informa- 
tion about the tests that is typically included in parent, student, and educator bro- 
chures. There is no need to duplicate information about the tests when that infor- 
mation is already available in an easily-accessible document. 

— Assessment reports and brochures (e.g., the informational and interpretive 

documents that describe the assessment program for parents, students, community 
members, and educators) should refer to the accountability system and reports. 
That is, after all, where the stakes for the assessment lie. 

■ Some stakeholders refer to report cards for specific analytical purposes, such as policy 
analysis, program evaluation, and needs analyses. Since the report card system can only 
accomplish so many things at once, consider: 
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“ Providing, upon request, electronic data files that include report card indicators or 
the data on which they are based (e.g., teacher-pupil ratio v. number of teachers 
and number of students) for evaluators and researchers; 

“ Producing briefs on policy issues and programs that are explicitiy linked to the 
information that appears in the report cards — ror link existing reports explicitiy to 
this information; and 

“ Producing briefs for specific audiences (e.g., the media) that succinctly summarize 
accountability information in two pages or less. 

It is key to ensure not only that accountability reports meet the requirements put forth in the 
No Child Left Behind Act of 2001 , but that they communicate to a wide variety of 
stakeholders the values, expectations, and achievements of the community’s students. The 
design of clear and succinct reports can accomplish this aim, and successfully drive stake- 
holders to action on behalf of these students. 



5 
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Appendix A 



Reporting Requirements 

This Appendix provides a series of tables describing reporting requirements under NCLB. 
Please refer back to the text for suggestions on how these indicators can be reported 

effectively. 
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A1 : NCLB Report Card Reporting Requirements 

Under the NCLB Act States and school districts (LEAs) are required, not later than the beginning of the 2002-03 school year, to 
prepare and disseminate annual report cards. Requirements for the data elements to be reported are set forth under section 
1 1 1 1 (h) of the law and summarized in this figure. Although schools are not required to prepare and disseminate their own 
reports, information about whether a school has been identified for improvement and how a school's students are achieving 
must be included in the State and school district reports. 

States must also report annually, beginning with school year 2002-03, to the Secretary of Education information on their 
progress related to requirements set forth in section 1111 (h)(4). 

States and school districts reports are required to be concise and presented in "an understandable and uniform format and, to 
the extent practicable, provided in a language that parents can understand" (sections 1 1 1 l(h)(l)(B)(ii) and 1111(h)(2)(E)). 
Additionally, school district reports must be made "widely available through public means such as posting on the Internet, 
distribution to the media..." (section 1111(h)(2)(E)). 
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AccountaWlltv Informotlon I 








1 


Comparison between the actual achievement for each 
group^'^ and the State's annual objectives for AYP: 

All Students 

Economically Sisadvantaged 
Major Racial and Ethnic groups 
Students with Disabilities 
Limited English Proficiency Students 


X 


X 


X 




Aggregate and disaggregate^ information on other 
academic indicators that the State has selected for AYP: 
Including graduation rate for secondary and an 
academic indicator for Elementary, Middle schools 


X 


X 


X 




Aggregate information on any additional Indicators the 
state may use to determine AYP 


X 


X 


X 




Performance of LEAs regarding achieving AYP 


X 


X 






Number of Schools Identified for improvement 


X 


X 




X 


Names of Schools In Improvement 


X 


X 




X 


Percentage of Schools identified for improvement 




X 






How long the Schools have been identified for 
improvement 




X 






Whether school has been identified for improvement 






X 




Reason(s) school was identified for improvement 






X 


X 


Measures taken to address achievement problems of 
Schools identified for Improvement 








X 
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state 


-LEA 


School 


Annual 




Report 


Report 


Report 


Report 




Cad 


Cord 


Cord 


to Sec. 


Items to be Reported 


inh(l)C 


llllh(2) 


1111(2)8 


llllh(4) 


Teacher Information^ 

Professional qualifications of teachers as defined by 
the state 


X 


X 


X 


X 


Percentage of teachers teaching with emergency or 
provisional credentials 


X 


X 


X 




Percentoge of classes taught by highly qualified 
teachers In the state, LEA, and school 








X 


Percentoge of classes In the state not taught by highly 
qualified teachers (aggregate arxJ In the highest and 
lowest quorttle schools based on poverty) 

English Language Proficiency 

Information on the acquisition of English proficiency 


X 


X 


X 

X 





by LEP students 

Supplemental Services ond School Choice 

Number of students and schools that participated in X 

supplemental services and public school choice 

Time Line — ■ 

Not later than the beginning of the Beginning 

2002-2003 SY with school 

year 

2002-2003 

’Except that such disaggregation shall not be required in a case in which the number of students in a category is insufficient to 
yield statistically reliable information or the results would reveal personally identifiable information about an individual student 

^Not required for Adequate Yearly Progress 

^In any year before the State begins to provide information described above, information on the results of student academic 

assessments 

^Including students who are enrolled in a school or LEA for a full academic year, as defined by the state. 

^Sec. 1119 a/b requires reports on schools, LEAs, and the state— regarding meeting measurable objectives related to the 
percentage of highly qualified teachers in core academic subjects: English, reading or language arts, mathematics, science, 

foreign languages, civics and government, economics, arts, histor y, and geography. 

Major racial~^d ethnic subgroups: African American/Black, Asian/Pacific Islander, American Indian/Alaskan Native, Hispanic, 
White 
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A2: Non Public Reports Required by the No Child Left Behind Act 



Individual student report (ISR) 

A single report that provides detailed infornnation to students, parents and teachers on the 
perfornnance of each student in each content area assessed. 



Sunnnnary Reports 

A series of reports at the clossroonn, school, district, and state level that provides sunnnnary 
statistics, such os overage scale scores and the percent scoring in each proficiency level 



Technical Reports 

Annual reports on the activities and analyses for the nnost recent previous testing cycle, 
including docunnentotion of the deveiopnnent and onoiyses processes for the nnost recent 
test odnninistrotion, and evidence in support of the technicoi quality of the test 
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A3: Highly Qualified Teacher as Defined in the NCLB Act 



. Highly Qualified Teacher Definition 

General Requirements ' : 

When used with respect to any public elementary school or secondary school teacher, 
refers to a teacher that— 

■ Has obtained full state certification, which may include certification by 
alternative routes; 

or 

■ Has passed the state licensure examination and holds a license to teach in the 
state; and 

■ Has not hod certification waived on on emergency, temporary or 
provisional basis 

Note: Teachers in public charter schools must meet the requirements of the state's 
public school charter low. 

Additional Requirements 
A New Elementary School Teacher. 

■ Bachelor's degree; and 

■ Pass o "rigorous" state test of subject matter knowledge and pedagogy in 
reading, writing, mathematics, and other subjects of the elementary school 
curriculum 

A New Middle or Secondary School Teacher: 

■ Bachelor's degree; and 

■ Pass o "rigorous" state test in each of the academic subjects in which the 
teacher teaches; 

or 

■ Successfully completing in each academic subject taught: 

— An undergraduate major 

— Graduate Degree 

— Coursework equivalent to an undergraduate major 
— Advanced certification or credentials 

An Elementary, Middle, or Secondary Teacher who Is not new to the profession: 

■ Bachelor's degree; and 

■ Meet the applicable requirements for teachers new to the profession; 
or 

■ Based on o high, objective, uniform State standards of evaluation demonstrates 
competency in each academic subject in which the teacher teaches^ 
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'A teacher meets the requirements listed if the teacher 

■ Has fulfiiied the State's certification and iicensure requirements appiicabie to the 
years of experience the teacher possesses or 

■ Is participating in an alternative route to certification program under which: 

— The teacher receives high-quality professional development that is sustained, 
intensive, and classroom focused in order to have a positive and lasting 
effect on classroom instruction, before and while teaching; 

— Participates in a program of intensive supervision that consists of structured 
guidance and regular ongoing support for teachers or a teacher mentoring 
program; 

— Assumes functions as a teacher only for a specified period of time not to 
exceed three years; 

— Demonstrates satisfactory progress toward full certification as prescribed by 
the state; 

— The state ensures, through its certification and licensure process, that the 
above provisions are met. 

^Demonstrated competence in all academic subjects the teacher teaches based on a 
high objective, uniform state standard of evaluation that: 

■ Is set by the state for grade appropriate academic subject matter and teaching 
skills 

■ Is aligned with state academic content and student achievement standards and 
developed in consultation with core content specialist, teachers, principals, and 
school administrators 

■ Provides objective, coherent information about the teacher's attainment of core 
content knowledge in the academic subject taught 

■ Is applied uniformly to all teachers in the same academic subject and grade level 
throughout the state 

■ Takes into consideration, but is not based primarily on, the time the teacher has 
been teaching the subject 

■ Is made available to the public upon request 

■ May involve multiple objective measure of teacher competency 

Sources: 

Title IX, Section 9101 of the Elementary and Secondary Education Act 
Title I, Section 1111 of the Elementary and Secondary Education Act 
Final Regulations, Title l-Improving the Academic Achievement of the Disadvantaged 
CCSSO, No Child Left Behind, A Description of State Responsibilities 

Note: Highly Qualified Teacher applies to all "core academic subjects": English, reading or language 
arts, mathematics, science, foreign languages, civic, government, economics, arts, history, and 
geography. 
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A4: Graduation Rate as Defined in the No Child Left Behind Act 

200. 1 9 Other Academic Indicators 

(a) Each state must use the following other academic indicators to determine AYP: 

(1 ) High Schools, (i) The graduation rate for public high schools, which means- 

(A) The percentage of students, measure from beginning high school, who graduate from 
high school with a regular diploma (not including an alternative degree that is not fully 
aligned with the State's academic standards, such os o certificate or o GED) in the 
standard number of years; or 

(B) Another definition, developed by the State and approved by the Secretary in the State 
plan, that more accurately measures the rote of students who graduate from high school 
with o regular diploma os defined above 

(ii) In defining the graduation rate, the State must ovoid counting o dropout as a transfer. 
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A5: Program Evaluation reports 



Report 


Due 


Reading First: Title I Part B Section 1202 (d) (5) (C) 


Annual for States 
Awarded Funds 


Migrant Students: Title I Part C Section 1308 (b)(4)(A&B) 


Due April 30, 2003 


Neglect and Delinquent Program: Title I Part D 


Once every 3 years 


Advanced Placement Programs: Title I Part G 


Annual 


Drop Out Prevention Programs: Title I Part H 


Annual, beginning 
after the first fiscal 
year an EA receives a 
grant 


Math and Science Partnerships: Title II Part B Section 2202 (f) 


Annual 


Language Instruction for LEP and Immigrant Children: 
Title III Section 3121 


Every Second Year 


Professional Development Grants: Title III Subpart 3 


Every 2 years (no 
report until funded 


Safe and Drug Free Schools: Title IV Part A 


December 1, 2003 
and every 2 years 
thereafter 


Innovative Programs: Title V Part A 


Annual 


Public Charter Schools: Title V Part B 


Annual 
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A6: Notice of Identification for Schools in Improvement, Corrective Action and 
Restructuring 

If an LEA Identifies a school for Improvement or subjects the school to corrective action or restructur- 
ing, the LEA must prorhptly notify the parents of each child enrolled In the school of this Identifica- 
tion. 

The notice must include the following: 

■ An explanation of what the identiflGation means, and how the school compares in terms of 
academic achievement to other elementary and secondary schools served by the LEA and 
SEA involved, 

■ The reason for identification. 

■ An explantation of how parents can become involved in addressing the academic issues that 
led to identification. 

■ An explanation of the parent's option to transfer their child to another public school, including 
the provision of transportation to the new school. 

— The explanation of a parents option to transfer must include, at minimum, information on 
the academic achievement of the school or schools to which the child may transfer; and 
— The explanation may include other information on the school or schools to which the child 
may transfer. 

If the school Is in Its second year of improvement or subject to corrective action or restructuring, a 
notice must be provided explaining supplemental educational services for their child. 

The annual notice of the availability of supplemental educational services must include at minimum: 

■ The identity of approved providers of those services available within the LEA, including providers 
of technology-based or distance-learning supplemental educational services, and providers 
that make services reasonably available in neighboring LEAs 

■ A brief description of the services, qualifications, and demonstrated effectiveness of the 
providers 

Information About Action Taken: 

An LEA must publish and disseminate to the parents of each student enrolled, and to the public 
information regarding any action taken by a school and the LEA to address the problems that led to 
the LEA'S Identification of the school for Improvement, corrective action, or restructuring. 

The information must include the following: 

■ An explanation of what the school is doing to address the problems of low achievement 

■ An explantation of what the LEA or SEA is doing to help the school address the problems of low 
achievement 
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Appendix C 

NCLB Reporting and Design Resources 

This Appendix provides a list of resources on design principles, the reporting requirements 
of NCLB, and others. Check the Council of Chief State School Officers’ web site, 
www.ccsso.org, for a frequently updated list of additional resources 

A-Plus Communications. (1999). Reporting results: What the public wants to know. [On- 
line] http:/ /www.ksaplus.com/ksa/EdWeek%20Results.pdf 

Bers, T. H., & Seybert, J. A. (2000). Effective reporting. Association for Institutional 
Research. 

Council of Chief State School Officers. (2002) No Child Left Behind Act: A Description 
of State Responsibilities. Washington, DC: CCSSO. [On-line] http://www.ccsso.org/pdfs/ 
NCLB2002.pdf 

Council of Chief State School Officers. (2001). Handbook for professional development in 
assessment literacy. Washington, DC: CCSSO. 

Council of Chief State School Officers, Accountability Web Page, http://www.ccsso.org/ 
accountability! .htm 

Diggle, PJ. Liang, K-Y and Zeger S.L. (1 994). Analysis of longitudinal data. Oxford Science 
Publications. 

Gong, B. (2002). Designing school accountability systems. Council of Chief State School 
Officers. [On-line] http:/ / www.ccsso.org/ pdfs/ designing_school_acct_syst.pdf 

Jaeger, R. M. & Tucker, C. G. (1 997). Analyzing, disaggregating, reporting, and interpreting 
students’ achievement test results: A guide to practice for Title I and beyond. Washington, 
DC: Council of Chief State School Officers. [On-line] http://www.ccsso.org/pdfs/ 
analyze.pdf 

Klass, G. Presenting Data: Tabular and graphic display of social indicators. [On-line] http:/ 
/lilt.ilstu.edu/gmklass/posl 38/ datadisplay/ 

KSA-Plus Communications. (2002, June). Hounded by bad news? Write your own headlines 
with your school accountability reports, (includes report examples) 
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Northern Illinois University, Division of Public Administration. Graphics and public 
management. [On-line] http:/ /www.niu.edu/pub_ad/p512/graphics2.htm 

Tufte, E. R. (1990). Envisioning information. Cheshire, CT: Graphics Press. 

Tufte, E. R. (2001). The visual display of quantitative information. Cheshire, CT: Graphics 
Press. 

Tufte, E. R. (1997). Visual explanations: Images and quantities, evidence and narrative. 
Cheshire, CT: Graphics Press. 

U.S. Department of Education. (2002). No Child Left Behind Web Site, 
www.nochildleftbehind.gov. 

Wainer H. Rounding tables. Chance, new directions for statistics and computing, 1 1 : 46 — 
50. 
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Copies of this report may be ordered from: 
Council of Chief State School Officers 
Attn: Publications 

One Massachusetts Ave., N.W., Suite 700 
Washington, DC 20001 
Phone:(202)336-7016 
Fax: (202) 408-8072 
Or go to www.ccsso.org/ publications 
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U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education (NLE) 
Educational Resources Information Center (ERIC) 
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